Deep Reservoir Computing: A Critical Analysis
نویسنده
چکیده
In this paper we propose an empirical analysis of deep recurrent neural networks (RNNs) with stacked layers. The analysis aims at the study and proposal of approaches to develop and enhance multiple timescale and hierarchical dynamics in deep recurrent architectures, within the efficient Reservoir Computing (RC) approach for RNN modeling. Results point out the actual relevance of layering and RC parameters aspects on the diversification of temporal representations in deep recurrent models.
منابع مشابه
Hierarchical Temporal Representation in Linear Reservoir Computing
Recently, studies on deep Reservoir Computing (RC) highlighted the role of layering in deep recurrent neural networks (RNNs). In this paper, the use of linear recurrent units allows us to bring more evidence on the intrinsic hierarchical temporal representation in deep RNNs through frequency analysis applied to the state signals. The potentiality of our approach is assessed on the class of Mult...
متن کاملDeep-ESN: A Multiple Projection-encoding Hierarchical Reservoir Computing Framework
As an efficient recurrent neural network (RNN) model, reservoir computing (RC) models, such as Echo State Networks, have attracted widespread attention in the last decade. However, while they have had great success with time series data [1], [2], many time series have a multiscale structure, which a single-hidden-layer RC model may have difficulty capturing. In this paper, we propose a novel hi...
متن کاملShort-term Memory of Deep RNN
The extension of deep learning towards temporal data processing is gaining an increasing research interest. In this paper we investigate the properties of state dynamics developed in successive levels of deep recurrent neural networks (RNNs) in terms of short-term memory abilities. Our results reveal interesting insights that shed light on the nature of layering as a factor of RNN design. Notic...
متن کاملDeep Echo State Network (DeepESN): A Brief Survey
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced deep Echo State Network (deepESN) model opened the way to an extremely efficient approach for designing deep neural networks for temporal data. At the same time, the study of deepESNs allowed...
متن کاملSource of Cement in the Great Oolite Reservoir, Storrington Oil Field, Weald Basin, South of England
The source of cement in oilfields is critical to the prediction of the distribution of cements in the reservoirs and also prediction of reservoir quality. The source of mineral forming cements has been determined for the Storrington oolitic carbonate reservoir (Middle Jurassic Great Oolite Formation, Weald Basin, onshore UK) using a combination of petrography, electron microscopy, fluid inclusi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016